Cruise-assist image generation device, cruise-assist image generation method, in-vehicle camera and equipment-control assist image generation device

ABSTRACT

A cruise-assist image generation device includes: a steering-angle information acquisition unit acquiring steering angle information of a steering wheel; an additional line generation unit generating a cruise assist additional line corresponding to the acquired steering angle information and formed by combination of a given number of segments; and an additional line superimposing unit superimposing the generated cruise assist additional line on a taken image of the vehicle periphery to obtain a composite image. The additional line generation unit generates respective curved lines forming part or all of the given number of segments by quadratic curve approximation using coordinate data corresponding to the acquired steering angle information and including coordinates of two endpoints and one middle point representing each segment, and generating respective straight lines forming the remains of the given number of segments by collinear approximation using coordinate data corresponding to the acquired steering angle information and including coordinates of two end points representing each segment.

CROSS REFERENCES TO RELATED APPLICATIONS

This Application is a Continuation Application of application Ser. No.15/363,710, filed Nov. 29, 2016, which is a Continuation Application ofapplication Ser. No. 13/856,663, filed Apr. 4, 2013, now U.S. Pat. No.9,533,709, issued on Jan. 3, 2017, which claims priority to JapanesePatent Application JP 2012-099500 filed in the Japan Patent Office onApr. 25, 2012, the entire contents of which are hereby incorporated byreference.

FIELD

The present disclosure relates to a cruise-assist image generationdevice, a cruise-assist image generation method, an in-vehicle cameraand an equipment-control assist image generation device, andparticularly relates to a cruise-assist image generation device and soon generating a composite image in which a cruise assist additional lineis superimposed on a taken image.

BACKGROUND

A technique is known in related art, in which cruise assist is realizedby displaying on a monitor a composite image in which a cruise assistadditional line indicating a cruise-estimation vehicle width track of avehicle is superimposed on a taken image of the vehicle periphery asdisclosed in, for example, JP-A-2002-330428 (Patent Document 1).

The cruise-estimation vehicle width track of the vehicle to be fixed byan individual vehicle design or a camera design eigenvalue differsaccording to vehicles, and is fixed in accordance with a steering angleof the vehicle at that time. Accordingly, the setting of the cruiseassist additional line (a dynamic guide line) corresponding to thesteering angle of the vehicle is performed by a microprocessor havingrelatively abundant ability in calculation processing such as a cameracontroller ECU or a navigation ECU.

SUMMARY

In the above system of the related-art configuration, an expensive ECUis indispensable between the camera and the monitor such as an LCD.Therefore, it is difficult to provide a display function of the dynamicguide line, which is highly convenient in driving/steering, forlow-price vehicles.

In view of the above, it is desirable to easily provide the displayfunction of the dynamic guide line, which is highly convenient indriving/steering, for low-price vehicles.

An embodiment of the present disclosure is directed to a cruise-assistimage generation device including a steering-angle informationacquisition unit acquiring steering angle information of a steeringwheel, an additional line generation unit generating a cruise assistadditional line corresponding to the acquired steering angle informationand formed by combination of a given number of segments, and anadditional line superimposing unit superimposing the generated cruiseassist additional line on a taken image of the vehicle periphery toobtain a composite image, in which the additional line generation unitgenerates respective curved lines forming part or all of the givennumber of segments by quadratic curve approximation using coordinatedata corresponding to the acquired steering angle information andincluding coordinates of two endpoints and one middle point representingeach segment, and generates respective straight lines forming theremains of the given number of segments by collinear approximation usingcoordinate data corresponding to the acquired steering angle informationand including coordinates of two end points representing each segment.

In the embodiment of the present disclosure, steering angle informationof the steering wheel is acquired by the steering-angle informationacquisition unit. In this case, the steering-angle informationacquisition unit may acquire steering angle information of the steeringwheel, for example, from an in-vehicle network. The cruise assistadditional line corresponding to the acquired steering angle informationand formed by combination of the given number of segments is generatedby the additional line generation unit. Then, the generated cruiseassist additional line is superimposed on the taken image of the vehicleperiphery by the additional line superimposing unit to obtain thecomposite image. The taken image may be a taken image obtained byimaging a back region of a vehicle or a side region of a vehicle.

Additionally, part of all of the given number of segments is formed bycurved lines, and the remains are formed by straight lines. The curvedlines are generated by quadratic curve approximation using coordinatedata corresponding to the steering angle information and includingcoordinates of two end points and one middle point representing eachsegment. The straight lines are formed by collinear approximation usingcoordinate data including coordinates of two end points representingeach segment.

As described above, in the embodiment of the present disclosure, thecruise assist additional line is formed by combination of the givennumber of segments and part or all of the given number of segments isformed by curved lines and the remains are formed by straight lines.Additionally, segments of curved lines are generated by quadratic curveapproximation using coordinate data of two end points and one middlepoint representing each segment, and segments of straight lines aregenerated by collinear approximation using coordinate data of two endpoints representing each segment.

Accordingly, the display function of the cruise assist additional linecorresponding to the steering angle of the steering wheel, namely, thedisplay function of the dynamic guide line can be realized by a simplecircuit configuration and a small capacity memory. That is, the displayfunction of the dynamic guide line can be realized without using anexpensive ECU. Accordingly, it is possible to easily provide the displayfunction of the dynamic guideline for low-price vehicles according tothe embodiment of the present disclosure.

In the embodiment of the present disclosure, one middle point may be apoint positioned in the middle between the two endpoints in thehorizontal direction or the vertical direction. Also in the embodimentof the present disclosure, part of end points of the given number ofsegments may be used in common. Accordingly, the number of coordinatedata for generating the given number of segments can be reduced andfurther saving of the memory can be realized.

Further, in the embodiment of the present disclosure, a coordinate datastorage unit storing the coordinate data may be further provided. Inthis case, the coordinate data storage unit may store coordinate datacorresponding to discrete values of steering angles of the steeringwheel, and the additional line generation unit may use the steeringangle indicated by the acquired steering angle information as a targetsteering angle and, when coordinate data corresponding to the targetsteering angle does not exist in the coordinate data storage unit, maygenerate and use coordinate data corresponding to the target steeringangle by interpolation processing of coordinate data corresponding tosteering angles in the vicinity of the target steering angle acquiredfrom the coordinate data storage unit. The simplest method of theinterpolation processing is linear interpolation processing usingneighboring coordinates. It is also possible to perform curveinterpolation in a higher order using quadratic curves and the like forobtaining smoother variation of steering angles.

Another embodiment of the present disclosure is directed to anin-vehicle camera including an imaging unit taking images of the vehicleperiphery, and a cruise-assist image generation unit, in whichcruise-assist image generation unit includes a steering-angleinformation acquisition unit acquiring steering angle information of asteering wheel, an additional line generation unit generating a cruiseassist additional line corresponding to the acquired steering angleinformation and formed by combination of a given number of segments, andan additional line superimposing unit superimposing the generated cruiseassist additional line on a taken image obtained in the imaging unit toobtain a composite image, in which the additional line generation unitgenerates respective curved lines forming part or all of the givennumber of segments by quadratic curve approximation using coordinatedata corresponding to the acquired steering angle information andincluding coordinates of two endpoints and one middle point representingeach segment, and generates respective straight lines forming theremains of the given number of segments by collinear approximation usingcoordinate data corresponding to the acquired steering angle informationand including coordinates of two end points representing each segment.

In the embodiment of the present disclosure, the in-vehicle cameraincludes the imaging unit taking images of the vehicle periphery and thecruise-assist image generation unit. The cruise-assist image generationunit includes the steering-angle information acquisition unit, theadditional line generation unit and the superimposing unit. Steeringangle information of the steering wheel is acquired by the steeringangle information acquisition unit. The cruise assist additional linecorresponding to the acquired steering angle information and formed bycombination of the given number of segments is generated by theadditional line generation unit. Then, the generated cruise assistadditional line is superimposed on the taken image obtained in theimaging unit to obtain a composite image by the additional linesuperimposing unit.

In this case, part of all of the given number of segments is formed bycurved lines, and the remains are formed by straight lines. The curvedlines are generated by quadratic curve approximation using coordinatedata corresponding to the steering angle information and includingcoordinates of two end points and one middle point representing eachsegment. The straight lines are formed by collinear approximation usingcoordinate data including coordinates of two end points representingeach segment.

As described above, in the embodiment of the present disclosure, thecruise assist additional line is formed by combination of the givennumber of segments and part or all of the given number of segments isformed by curved lines and the remains are formed by straight lines.Additionally, segments of curved lines are generated by quadratic curveapproximation using coordinate data of two end points and one middlepoint representing each segment, and segments of straight lines aregenerated by collinear approximation using coordinate data of two endpoints representing each segment.

Accordingly, the display function of the cruise assist additional linecorresponding to the steering angle of the steering wheel, namely, thedisplay function of the dynamic guide line can be realized by a simplecircuit configuration and a small capacity memory. That is, thein-vehicle camera having the display function of the dynamic guide linecan be formed at low costs. Accordingly, it is possible to easilyprovide the display function of the dynamic guideline for low-pricevehicles by using the in-vehicle camera according to the embodiment ofthe present disclosure.

Still another embodiment of the present disclosure is directed to acruise-assist image generation device including an additional linegeneration unit generating a cruise assist additional line formed bycombination of a given number of segments, a coordinate data storageunit storing coordinate data necessary for generating the segments, andan additional line superimposing unit superimposing the generated cruiseassist additional line on a taken image of the vehicle periphery toobtain a composite image, in which the additional line generation unitgenerates respective curved lines forming part or all of the givennumber of segments by quadratic curve approximation using coordinatedata of two end points and one middle point representing each segment.

Yet another embodiment of the present disclosure is directed to anequipment-control assist image generation device including a devicemonitoring external peripheral environment by an imaging device mountedon driving equipment and capable of superimposing a track on an image inwhich a coordinate change occurs along with a driving operation on theimage obtained by the device, an additional line generation unitgenerating a track assist additional line corresponding to acquiredsteering angle information and formed by combination of a given numberof segments, and an additional line superimposing unit superimposing thegenerated track assist additional line on a background image to obtain acomposite image, in which the additional line generation unit generatesrespective curved lines forming part or all of the given number ofsegments by quadratic curve approximation using coordinate datacorresponding to the acquired steering angle information and includingcoordinates of two end points and one middle point representing eachsegment, and generates respective straight lines forming the remains ofthe given number of segments by collinear approximation using coordinatedata corresponding to the acquired steering angle information andincluding coordinates of two end points representing each segment.

Still yet another embodiment of the present disclosure is directed to acruise-assist image generation device including an imaging unit equippedwith a variable function of a visual field range, an additional linegeneration unit generating a cruise assist additional line formed bycombination of a given number of segments, and an additional linesuperimposing unit superimposing the generated cruise assist additionalline on a taken image of the vehicle periphery to obtain a compositeimage, in which the additional line generation unit has a unitconfigured to generate respective curved lines forming part or all ofthe given number of segments by quadratic curve approximation usingcoordinate data corresponding to acquired steering angle information andincluding coordinates of two end points and one middle pointrepresenting each segment, generate respective straight lines formingthe remains of the given number of segments by collinear approximationusing coordinate data corresponding to the acquired steering angleinformation and including coordinates of two end points representingeach segment, and update and generating coordinate data necessary forgenerating the additional line in accordance with a variable state ofthe visual field range.

According to the embodiments of the present disclosure, the displayfunction of the dynamic guide line which is highly convenient indriving/steering can be provided for low-price vehicles.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration example of anin-vehicle camera according to an embodiment;

FIG. 2 is a view showing an example of a cruise assist additional linedisplayed at the time of reversing a vehicle;

FIG. 3 is a chart showing an example of a set of coordinate datacorresponding to one steering angle to be stored in a lookup table;

FIG. 4 is a chart schematically showing the entire coordinate data to bestored in the lookup table so as to correspond to respective discretesteering angles;

FIG. 5 is a view showing another example of the cruise assist additionalline displayed at the time of reversing the vehicle;

FIG. 6 is a chart showing another example of a set of coordinate datacorresponding to one steering angle to be stored in the lookup table;

FIG. 7 is a view showing another example of the cruise assist additionalline displayed at the time of reversing the vehicle;

FIG. 8 is a chart showing another example of a set of coordinate datacorresponding to one steering angle to be stored in the lookup table;

FIG. 9 is a flowchart showing an example of processing procedures in asteering-angle information extraction unit, a steering anglecorresponding coordinate acquisition unit and a cruise-assist additionalline generation unit at given time intervals;

FIG. 10 is a view showing an example of a taken image of a side regionof a vehicle on which a cruise assist additional line is superimposed;

FIG. 11 is a view showing an example of the cruise assist additionalline superimposed on the taken image of the side region of the vehicle;and

FIG. 12 is a view showing an example of a set of coordinate datacorresponding to one steering angle to be stored in the lookup table.

DETAILED DESCRIPTION

Hereinafter, a mode for carrying out the present disclosure (hereinafterreferred to as an “embodiment”) will be explained. The explanation willbe made in the following order.

1. Embodiment

2. Modification Example

<1. Embodiment> [Configuration Example of In-Vehicle Camera]

FIG. 1 shows a configuration example of an in-vehicle camera 100according to an embodiment. The in-vehicle camera 100 is a back cameradisposed at the back of the vehicle, a side camera disposed at the sideof the vehicle on a passenger's seat side or any other camera. The backcamera is applied in the embodiment.

The in-vehicle camera 100 includes a steering-angle informationextraction unit 101, a steering-angle corresponding coordinateacquisition unit 102, a look up table (LUT) 103, a cruise-assistadditional line generation unit 104, an imaging unit 105, anadditional-line superposition unit 106 and an output terminal 107. Thesteering-angle information extraction unit 101 extracts steering-angleinformation of a steering wheel 10 from an in-vehicle network. Thesteering-angle information extraction unit 101 forms a steering-angleinformation acquisition unit.

The look up table (LUT) 103 stores coordinate data corresponding torespective discrete values of steering angles of the steering wheel 10.A cruise assist additional line superimposed on the taken image to bedisplayed is formed by combination of a given number of segments. Partor all of the given number of segments is formed by curved lines and theremains are formed by straight lines. The look up table 103 storescoordinate data for directly or indirectly obtaining coordinate data ofpoints representing the respective given number of segments so as tocorrespond to respective discrete values of steering angles. Here,points representing a curved line are two end points and one centerpoint. Points representing a straight line are two end points.

FIG. 2 shows an example of the cruise assist additional line displayedat the time of reversing the vehicle. The cruise assist additional lineincludes nine segments. The nine segments includes a first horizontalline L1, a first vertical-left line L2, a first vertical-right line L3,a second horizontal line L4, a second vertical-left line L5, a secondvertical-right line L6, a third horizontal line L7, a thirdvertical-left line L8 and a third vertical-right line L9. In theexample, all nine segments are formed by curved lines.

In the first horizontal line L1, coordinates of a left end point arerepresented as (Xhc1−Xhw1, Yhl1), coordinates of a right end point arerepresented as (Xhc1+Xhw1, Yhr1) and coordinates of a center point arerepresented as (Xhc1, Yhc1). In this case, the center point ispositioned at the center of right-and-left two end points in thehorizontal direction, two end points and one center point arerepresented as five coordinate data of Xhw1, Xhc1, Yhl1, Yhc1 and Yhr1.Accordingly, coordinate data stored in the lookup table (LUT) 103 isreduced from 6 to 5. The same applies to other segments.

Concerning suffixes of horizontal components, “h” denotes a horizontalline, “c” denotes a center point, “w” denotes a width in the horizontaldirection, “l” denotes a left point and “r” denotes a right point,respectively. Concerning suffixes of vertical components, “v” denotes avertical line, “m” denotes a “middle point”, “h” denotes a height in thevertical direction, “b” denotes a bottom point and “t” denotes a toppoint.

In the first vertical-left line L2, coordinates of a bottom end pointare represented as (Xvlb1, Yvlm1−Yvlh1), coordinates of a top end pointare represented as (Xvlt1, Yvlm1+Yvlh1) and coordinates of the middlepoint are represented as (Xvlt1, Yvlm1). In the first vertical-rightline L3, coordinates of a bottom end point are represented as (Xvrb1,Yvrm1−Yvrh1), coordinates of a top end point are represented as (Xvrt1,Yvrm1+Yvrh1) and coordinates of a middle point are represented as(Xvrm1, Yvrm1). The second horizontal line L4, the second vertical-leftline L5, the second vertical-right line L6, the third horizontal lineL7, the third vertical-left line L8 and the third vertical-right line L9are represented in the same manner as the above, though the detaileddescription is omitted.

FIG. 3 collectively shows a set of coordinate data including 45 (3×15)coordinates corresponding to one steering angle relating to the cruiseassist additional line shown in FIG. 2 to be stored in the lookup table103. FIG. 4 schematically shows the entire coordinate data correspondingto respective discrete steering angles to be stored in the lookup table103. The set of coordinate data corresponding to a steering angle Nincludes coordinate data of 45 coordinates from a coordinate (N, 1) to acoordinate (N, 45).

FIG. 5 shows another example of the cruise assist additional linedisplayed at the time of reversing the vehicle. The cruise assistadditional line also includes nine segments. The nine segments includesthe first horizontal line L1, the first vertical-left line L2, the firstvertical-right line L3, the second horizontal line L4, the secondvertical-left line L5, the second vertical-right line L6, the thirdhorizontal line L7, a third vertical-left line L8′ and a thirdvertical-right line L9′. In the example, seven segments are formed bycurved lines and the remaining two segments are formed by straightlines. The first horizontal line L1, the first vertical-left line L2,the first vertical-right line L3, the second horizontal line L4, thesecond vertical-left line L5, the second vertical-right line L6 and thethird horizontal line L7 are formed by curved lines in the same manneras the example of FIG. 2. The third vertical-left line L8′ and the thirdvertical-right line L9′ are formed by straight lines.

In the third vertical-left line L8′, coordinates of a bottom end pointare represented as (Xvlb3, Yvlb3) and coordinates of a top end point arerepresented as (Xvlt3, Yvlt3), which are represented by four coordinatedata of Xvlb3, Yvlb3, Xvlt3 and Yvlt3. In the third vertical-right lineL9′, coordinates of a bottom end point are represented as (Xvrb3, Yvrb3)and a top end point are represented as (Xvrt3, Yvrt3), which arerepresented by four coordinate data of Xvrb3, Yvrb3, Xvrt3 and Yvrt3.

FIG. 6 collectively shows a set of coordinate data including 43(=2×15+1×13) coordinates corresponding to one steering angle relating tothe cruise assist additional line shown in FIG. 5 to be stored in thelookup table 103.

FIG. 7 shows further another example of the cruise assist additionalline displayed at the time of reversing the vehicle. In the example, oneend point or both endpoints of each segment are used in common with endpoints of other segments, which can further reduce coordinate data to bestored in the lookup table (LUT) 103.

The cruise assist additional line also includes nine segments. The ninesegments includes a first horizontal line L1 a, a first vertical-leftline L2 a, a first vertical-right line L3 a, a second horizontal line L4a, a second vertical-left line L5 a, a second vertical-right line L6 a,a third horizontal line L7 a, a third vertical-left line L8 a and athird vertical-right line L9 a. In the example, seven segments areformed by curved lines and the remaining two segments are formed bystraight lines in the same manner as the example shown in FIG. 5. Thefirst horizontal line L1 a, the first vertical-left line L2 a, the firstvertical-right line L3 a, the second horizontal line L4 a, the secondvertical-left line L5 a, the second vertical-right line L6 a and thethird horizontal line L7 a are formed by curved lines. The thirdvertical-left line L8 a and the third vertical-right line L9 a areformed by straight lines.

In the first horizontal line L1 a, coordinates of a left end point arerepresented as (Xhc1−Xhw1, Yvlm1−Yvlh1), coordinates of a right endpoint are represented as (Xhc1+Xhw1, Yvrm1−Yvrh1) and coordinates of acenter point are represented as (Xhc1, Yhc1). In the second horizontalline L4 a, coordinates of a left endpoint are represented as (Xhc2−Xhw2,Yvlm2−Yvlh2), coordinates of a right end point are represented as(Xhc2+Xhw2, Yvrm2−Yvrh2) and coordinates of a center point arerepresented as (Xhc2, Yhc2). In the third horizontal line L7 a,coordinates of a left endpoint are represented as (Xhc3−Xhw3,Yvlm2+Yvlh2), coordinates of a right end point are represented as(Xhc3+Xhw3, Yvrm2+Yvrh2) and coordinates of a center point as (Xhc3,Yhc3).

In the first vertical-left line L2 a, coordinates of a bottom end pointare represented as (Xhc1−Xhw1, Yvlm1−Yvlh1) which are used in commonwith the left end point of the first horizontal line L1 a, coordinatesof an upper end point are represented as (Xhc2−Xhw2, Yvlm2−Yvlh2) whichare used in common with the left end point of the second horizontal lineL4 a and coordinates of a middle point are shown as (Xvlm1, Yvlm1). Inthe first vertical-right line L3 a, coordinates of a bottom end pointare represented as (Xhc1+Xhw1, Yvrm1−Yvrh1) which are used in commonwith the right end point of the first horizontal line L1 a, coordinatesof a top end point are represented as (Xhc2+Xhw2, Yvrm2−Yvrh2) which areused in common with the right end point of the second horizontal line L4a and coordinates of a middle point are represented as (Xvrm1, Yvrm1).

In the second vertical-left line L5 a, coordinates of a bottom end pointare represented as (Xch2−Xhw2, Yvlm2−Yvlh2) which are used in commonwith the left end point of the second horizontal line L4 a, coordinatesof a top end point are represented as (Xhc3−Xhw3, Yvlm2+Yvlh2) which areused in common with the left end point of the third horizontal line L7 aand coordinates of a middle point are represented as (Xvlm2, Yvlm2). Inthe second vertical-right line L6 a, coordinates of a bottom end pointare represented as (Xhc2+Xhw2, Yvrm2−Yvrh2) which are used in commonwith the right end point of the second horizontal line L4 a, coordinatesof a top end point are represented as (Xhc3+Xhw3, Yvrm2+Yvrh2) which areused in common with the right end point of the third horizontal line L7a and coordinates of a middle point are represented as (Xvrm2, Yvrm2).

In the third vertical-left line L8 a, coordinates of a bottom end pointare represented as (Xhc3−Xhw3, Yvlm2+Yvlh2) which are used in commonwith the left end point of the third horizontal line L7 a, andcoordinates of a top end point are represented as (Xvlt3, Yvlt3). In thethird vertical-right line L9 a, coordinates of a bottom end point arerepresented as (Xhc3+Xhw3, Yvrm2+Yvrh2) which are used in common withthe right end point of the third horizontal line L7 a, and coordinatesof a top end point are represented as (Xvrt3, Yvrt3).

FIG. 8 collectively shows a set of coordinate data including 25coordinates corresponding to one steering angle relating to the cruiseassist additional line shown in FIG. 7 to be stored in the lookup table103. As one end point or both end points of each segment are used incommon with end points of other segments, coordinate data to be storedin the lookup table (LUT) 103 is further reduced. That is, coordinatedata including 45 coordinates is necessary in the example shown in FIG.2 in which data is not used in common, however, only coordinate dataincluding 25 coordinates is necessary in the example of FIG. 7 in whichdata is used in common.

Returning to FIG. 1, the steering-angle corresponding coordinateacquisition unit 102 acquires coordinate data (coordinate data set) ofpoints representing respective segments forming the above cruise assistadditional line corresponding to the steering angle indicated bysteering angle information extracted in the steering-angle informationextraction unit 101. Here, when the steering angle indicated by thesteering angle information is a target steering angle and coordinatedata corresponding to the target steering angle exists in the lookuptable 103, the steering-angle corresponding coordinate acquisition unit102 directly acquires coordinate data corresponding to the targetsteering angle from the lookup table 103.

On the other hand, when coordinate data corresponding to the targetsteering angle does not exist in the lookup table 103, thesteering-angle corresponding coordinate acquisition unit 102 acquirescoordinate data corresponding to steering angles in the vicinity of thetarget steering angle from the lookup table 103 to generate coordinatedata corresponding to the target steering angle by linear interpolationprocessing. The cruise assist additional line (cruise prediction line)with respect to the steering angle of the vehicle is monotoneincreasing/decreasing, which can keep continuity by neighborinterpolation processing.

The cruise-assist additional line generation unit 104 generates thecruise assist additional line corresponding to a present steering anglebased on coordinate data (coordinate data set) acquired by thesteering-angle corresponding coordinate acquisition unit 102. In thiscase, a segment (a curved line) included in the cruise assist additionalline is generated by quadratic curve approximation using coordinate dataof two end points and one middle point. A segment (straight line)included in the cruise assist additional line is generated by collinearapproximation using coordinate data of two end points.

The imaging unit 105 obtains a taken image by taking an image of thevehicle periphery, that is, a back region of the vehicle in theembodiment. The imaging unit 105 is formed by, for example, a CMOS imagesensor and the like. The additional-line superposition unit 106superimposes the cruise assist additional line generated by thecruise-assist additional line generation unit 104 on the taken imageobtained by the imaging unit 105 to thereby obtain a composite image.Specifically, the additional-line superposition unit 106 combines animaging signal outputted from the imaging unit 105 with a display signalof the cruise assist additional line obtained by the cruise-assistadditional line generation unit 104, outputting an imaging signal aftercomposition.

The output terminal 107 derives the imaging signal obtained aftercombining the display signal of the cruise assist additional lineobtained by the additional-line superposition unit 106. To the outputterminal 107, for example, the monitor 20 formed by an LCD and so on isconnected.

The operation of the in-vehicle camera 100 shown in FIG. 1 will beexplained. In the steering-angle information extraction unit 101,steering angle information of the steering wheel 10 is extracted fromthe in-vehicle network. The steering angle information is supplied tothe steering-angle corresponding coordinate acquisition unit 102. Thesteering-angle corresponding coordinate acquisition unit 102 acquirescoordinate data (coordinate data set) of points representing respectivesegments forming the above-described cruise assist additional line,which corresponds to the steering angle indicated by the steering angleinformation extracted in the steering-angle information extraction unit101. The coordinate data is supplied to the cruise-assist additionalline generation unit 104.

In the cruise-assist additional line generation unit 104, the cruiseassist additional line corresponding to the present steering angle isgenerated based on the coordinate data (coordinate data set) acquired bythe steering-angle corresponding coordinate acquisition unit 102. Inthis case, a segment (a curved line) included in the cruise assistadditional line is generated by quadratic curve approximation usingcoordinate data of two end points and one middle point. A segment(straight line) included in the cruise assist additional line isgenerated by collinear approximation using coordinate data of two endpoints. The cruise assist additional line is supplied to theadditional-line superposition unit 106.

The taken image obtained by taking the image of the back region of thevehicle in the imaging unit 105 is supplied to the additional-linesuperposition unit 106. In the additional-line superposition unit 106,the cruise assist additional line generated by the cruise-assistadditional line generation unit 104 is superimposed on the taken imageobtained in the imaging unit 105 to thereby obtain a composite image.That is, in the additional-line superposition unit 106, the displaysignal of the cruise assist additional line obtained by thecruise-assist additional line generation unit 104 is combined with theimaging signal outputted from the imaging unit 105, and the imagingsignal after composition is outputted.

The imaging signal after composition obtained in the additional-linesuperposition unit 106 is supplied to the output terminal 107 as anoutput of the in-vehicle camera 100. When the monitor 20 is connected tothe output terminal 107, the imaging signal after composition issupplied to the monitor 20 from the output terminal 107. On the monitor20, a taken image on which the cruise assist additional line issuperimposed is displayed.

The extraction of steering angle information of the steering wheel 10 inthe steering-angle information extraction unit 101 is executedrepeatedly at given time intervals. When a value of the steering angleindicated by the steering angle information is different from a value ofthe previous time, coordinate data corresponding to the changed steeringangle is acquired in the steering-angle corresponding coordinateacquisition unit 102, and the cruise assist additional line is generatedin the cruise-assist additional line generation unit 104 so as tocorrespond to the changed steering angle. Therefore, the cruise assistadditional line supplied from the cruise-assist additional linegeneration unit 104 to the additional-line superposition unit 106 isupdated in accordance with the change in the steering angle of thesteering wheel 10. Accordingly, the cruise assist additional linesuperimposed on the taken image is dynamically updated in accordancewith the change of the steering angle of the steering wheel 10.

A flowchart of FIG. 9 shows an example of processing procedures in thesteering-angle information extraction unit 101, the steering-anglecorresponding coordinate acquisition unit 102 and the cruise-assistadditional line generation unit 104 performed at given time intervals.In Step ST1, the processing is started. Next, in Step ST2, steeringangle information Sa (t) at a time “t” is extracted from the in-vehiclenetwork in the steering-angle information extraction unit 101.

Next, in Step ST3, whether the steering angle information Sa (t)extracted this time is different from steering angle information Sa(t-1) extracted last time is determined. That is, whether a condition of|Sa(t)-Sa(t-1)|>0 is satisfied or not is determined in Step ST3. Whenthe extracted steering angle information is the same as informationextracted last time, the processing is ended immediately in Step ST6.

On the other hand, when the extracted steering angle information differsfrom the information extracted last time in Step ST3, coordinate datafor generating respective segments forming the cruise assist additionalline corresponding to the extracted steering angle information isacquired in the steering-angle corresponding coordinate acquisition unit102 in Step ST4. Then, the cruise assist additional line correspondingto the steering angle information extracted this time is generated basedon the newly-acquired coordinate data in the cruise-assist additionalline generation unit 104 in Step ST5. After that, the processing isended in Step ST6.

As described above, in the in-vehicle camera 100 shown in FIG. 1, thecruise assist additional line is formed by combination of the givennumber of segments and part or all of the given number of segments isformed by curved lines and the remains are formed by straight lines.Additionally, segments of curved lines are generated by quadratic curveapproximation using coordinate data of two end points and one middlepoint representing each segment, and segments of straight lines aregenerated by collinear approximation using coordinate data of two endpoints representing each segment.

Accordingly, the display function of the cruise assist additional linecorresponding to the steering angle of the steering wheel 10, namely,the display function of the dynamic guideline can be realized by asimple circuit configuration and a small capacity memory. That is, thein-vehicle camera 100 having the display function of the dynamic guideline can be formed at low costs. Accordingly, it is possible to easilyprovide the display function of the dynamic guideline for low-pricevehicles by using the in-vehicle camera 100.

<2. Modification Example>

The example of the back camera disposed at the back of the vehicle hasbeen shown in the above embodiment. However, the present disclosure canbe also applied in the same manner to a side camera disposed at the sideof the vehicle, for example, on a passenger's seat side of a vehicle,which images a side region of the vehicle and obtains a taken image toobtain the same advantages.

FIG. 10 shows an example of a taken image of a side region of a vehicleon which the cruise assist additional line is superimposed. The cruiseassist additional line in this example includes four segments. In thiscase, the lookup table (LUT) 103 stores coordinate data of pointsrepresenting the four segments, which corresponds to respective discretevalues of steering angles of the steering wheel 10. The cruise assistadditional line includes four segments of a first vertical-left line L1,a first horizontal line L2, a second vertical-left line L3 and a secondhorizontal line 14 as shown in FIG. 11. In this example, all foursegments are formed by curved lines.

In the first vertical-left line, coordinates of a bottom end point arerepresented as (Xvlb1, Yvlm1−Yvlh1), coordinates of a top end point arerepresented as (Xvlt1, Yvlm1+Yvlh1) and coordinates of a middle pointare represented as (Xvlb1, Yvlm1). In the second horizontal line L2,coordinates of a left endpoint are represented as (Xhc1−Xhw1, Yhl1),coordinates of a right end point are represented as (Xhc1+Xhw1, Yhr1)and coordinates of a center point are represented as (Xhc1, Yhc1).

In the second vertical-left line L3, coordinates of a bottom end pointare represented as (Xvlb2, Yvlm2−Yvlh2), coordinates of a top end pointare represented as (Xvlt2, Yvlm2+Yvlh2) and coordinates of a middlepoint are represented as (Xvlm2, Yvlm2). In the second horizontal lineL4, coordinates of a left end point are represented as (Xhc2−Xhw2,Yhl2), coordinates of a right end point are represented as (Xhc2+Xhw2,Yhr2) and coordinates of a center point are represented as (Xhc2, Yhc2).

FIG. 12 collectively shows a set of coordinate data including 20 (2×10)coordinates corresponding to one steering angle relating to the cruiseassist additional line shown in FIG. 11 to be stored in the lookup table103.

The operation interlocked with the steering has been explained in theabove embodiment. However, a track line generated in the presentdisclosure is not limited to the line generated by the operationinterlocked with the steering. In the technology of the presentdisclosure, stationary display curves can be realized by a small numberof setting coordinates, therefore, the technology of the presentdisclosure is effective for saving the memory also when drawing a fixedcruise line.

The case where the technology of the present disclosure is mainly usedfor reversing the vehicle or performing parallel parking the vehicle hasbeen shown in the above embodiment. However, the technology of thepresent disclosure can be also used for displaying a prediction trackfor movement of a motorboat and the like, a prediction track of aturning range in operation of heavy equipment such as a crane and aconstruction machine.

A variable function of a visual field of the imaging unit is notmentioned in the above embodiment. However, the imaging unit may beequipped with the variable function of the visual field. When the visualfield of the imaging unit varies, it is necessary to change the cruiseassist additional line to be superimposed on the taken image inaccordance with a variable state of the visual field. In this case, aunit configured to update and generating coordinate data necessary forgenerating the additional line in accordance with the variable state ofthe visual field is provided to solve the problem.

The present disclosure can also apply the following configurations.

(1) A cruise-assist image generation device including

a steering-angle information acquisition unit acquiring steering angleinformation of a steering wheel,

an additional line generation unit generating a cruise assist additionalline corresponding to the acquired steering angle information and formedby combination of a given number of segments, and

an additional line superimposing unit superimposing the generated cruiseassist additional line on a taken image of the vehicle periphery toobtain a composite image,

in which the additional line generation unit generates respective curvedlines forming part or all of the given number of segments by quadraticcurve approximation using coordinate data corresponding to the acquiredsteering angle information and including coordinates of two endpointsand one middle point representing each segment, and generates respectivestraight lines forming the remains of the given number of segments bycollinear approximation using coordinate data corresponding to theacquired steering angle information and including coordinates of two endpoints representing each segment.

(2) The cruise-assist image generation device described in the above(1),

in which one middle point is a point positioned in the middle betweenthe two end points in the horizontal direction or the verticaldirection.

(3) The cruise-assist image generation device described in the above (1)or (2),

in which part of endpoints of the given number of segments is used incommon.

(4) The cruise-assist image generation device described in any of theabove (1) to (3),

in which the taken image is a taken image to be obtained by imaging aback region of a vehicle.

(5) The cruise-assist image generation device described in any of theabove (1) to (3),

in which the taken image is a taken image to be obtained by imaging aside region of a vehicle.

(6) The cruise-assist image generation device described in any of theabove (1) to (5),

in which the steering-angle information acquisition unit acquiressteering angle information of the steering wheel from an in-vehiclenetwork.

(7) The cruise-assist image generation device described in any of theabove (1) to (6), further including

a coordinate data storage unit storing the coordinate data.

(8) The cruise-assist image generation device described in the above(7),

in which the coordinate data storage unit stores coordinate datacorresponding to discrete values of steering angles of the steeringwheel, and

the additional line generation unit uses the steering angle indicated bythe acquired steering angle information as a target steering angle and,when coordinate data corresponding to the target steering angle does notexist in the coordinate data storage unit, generates and uses coordinatedata corresponding to the target steering angle by interpolationprocessing of coordinate data corresponding to steering angles in thevicinity of the target steering angle acquired from the coordinate datastorage unit.

(9) The cruise-assist image generation device described in the above(8),

in which coordinate data corresponding to the target steering angle isgenerated by linear interpolation of coordinate data corresponding tosteering angles in the vicinity of the target steering angle acquiredfrom the coordinate data storage unit and the generated coordinate datais used.

(10) A cruise-assist image generation method including

acquiring steering angle information of a steering wheel,

generating a cruise assist additional line corresponding to the acquiredsteering angle information and formed by combination of a given numberof segments, and

superimposing the generated cruise assist additional line on a takenimage of the vehicle periphery to obtain a composite image,

in which respective curved lines forming part or all of the given numberof segments are generated by quadratic curve approximation usingcoordinate data corresponding to the acquired steering angle informationand including coordinates of two end points and one middle pointrepresenting each segment, and respective straight lines forming theremains of the given number of segments are generated by collinearapproximation using coordinate data corresponding to the acquiredsteering angle information and including coordinates of two end pointsrepresenting each segment in the process of generating the cruise assistadditional line.

(11) An in-vehicle camera including

an imaging unit taking images of the vehicle periphery, and

a cruise-assist image generation unit,

in which cruise-assist image generation unit includes

a steering-angle information acquisition unit acquiring steering angleinformation of a steering wheel,

an additional line generation unit generating a cruise assist additionalline corresponding to the acquired steering angle information and formedby combination of a given number of segments, and

an additional line superimposing unit superimposing the generated cruiseassist additional line on a taken image obtained in the imaging unit toobtain a composite image,

in which the additional line generation unit generates respective curvedlines forming part or all of the given number of segments by quadraticcurve approximation using coordinate data corresponding to the acquiredsteering angle information and including coordinates of two endpointsand one middle point representing each segment, and generates respectivestraight lines forming the remains of the given number of segments bycollinear approximation using coordinate data corresponding to theacquired steering angle information and including coordinates of two endpoints representing each segment.

(12) A cruise-assist image generation device including

an additional line generation unit generating a cruise assist additionalline formed by combination of a given number of segments,

a coordinate data storage unit storing coordinate data necessary forgenerating the segments, and

an additional line superimposing unit superimposing the generated cruiseassist additional line on a taken image of the vehicle periphery toobtain a composite image,

in which the additional line generation unit generates respective curvedlines forming part or all of the given number of segments by quadraticcurve approximation using coordinate data of two end points and onemiddle point representing each segment.

(13) An equipment-control assist image generation device including

a device monitoring external peripheral environment by an imaging devicemounted on driving equipment and capable of superimposing a track on animage in which a coordinate change occurs along with a driving operationon the image obtained by the device,

an additional line generation unit generating a track assist additionalline corresponding to acquired steering angle information and formed bycombination of a given number of segments, and

an additional line superimposing unit superimposing the generated trackassist additional line on a background image to obtain a compositeimage,

in which the additional line generation unit generates respective curvedlines forming part or all of the given number of segments by quadraticcurve approximation using coordinate data corresponding to the acquiredsteering angle information and including coordinates of two endpointsand one middle point representing each segment, and generates respectivestraight lines forming the remains of the given number of segments bycollinear approximation using coordinate data corresponding to theacquired steering angle information and including coordinates of two endpoints representing each segment.

(14) A cruise-assist image generation device including

an imaging unit equipped with a variable function of a visual fieldrange,

an additional line generation unit generating a cruise assist additionalline formed by combination of a given number of segments, and

an additional line superimposing unit superimposing the generated cruiseassist additional line on a taken image of the vehicle periphery toobtain a composite image,

in which the additional line generation unit has a unit configured to

generate respective curved lines forming part or all of the given numberof segments by quadratic curve approximation using coordinate datacorresponding to acquired steering angle information and includingcoordinates of two endpoints and one middle point representing eachsegment,

generate respective straight lines forming the remains of the givennumber of segments by collinear approximation using coordinate datacorresponding to the acquired steering angle information and includingcoordinates of two end points representing each segment and

update and generating coordinate data necessary for generating theadditional line in accordance with a variable state of the visual fieldrange.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image signal processing apparatus comprising: a steering angleinformation receiving circuitry configured to receive steering angleinformation from an in-vehicle network; and an image processingcircuitry configured to retrieve a data set from a plurality of datasets based on the steering angle information that is received, each dataset of the plurality of data sets including coordinate datacorresponding to a different steering angle of a plurality of steeringangles, generate a guideline formed by a combination of a number offirst direction segments and a number of second direction segments basedon the data set retrieved from the plurality of data sets, superimposethe guideline on an image of a vehicle periphery, output the image withthe guideline that has been superimposed to a display, and update theguideline based on a change of the steering angle information, wherein afirst end point coordinate of a first segment of the number of firstdirection segments is the same coordinate as a first end pointcoordinate of a first segment of the number of second directionsegments, wherein a second end point coordinate of the first segment ofthe number of first direction segments is the same coordinate as a firstend point coordinate of a second segment of the number of firstdirection segments, and wherein a second end point coordinate of thefirst segment of the number of second direction segments is the samecoordinate as a first end point coordinate of a second segment of thenumber of second direction segments.
 2. The image signal processingapparatus according to claim 1, wherein, to acquire the steering angleinformation from the in-vehicle network, the steering angle informationreceiving circuitry is further configured to acquire, repeatedly at atime interval, the steering angle information of a steering wheel. 3.The image signal processing apparatus according to claim 1, furthercomprising: a memory configured to store the plurality of data sets,each data set of the plurality of data sets includes coordinate datacorresponding to the different steering angle of the plurality ofsteering angles.
 4. The image signal processing apparatus according toclaim 3, wherein the memory is further configured to store thecoordinate data of the each data set of the plurality of data setscorresponding to a discrete value associated with the different steeringangle of the plurality of steering angles, and wherein the steeringangle information receiving circuitry is further configured to receivethe steering angle information of a target steering angle, and generatetarget coordinate data corresponding to the target steering angle byinterpolation processing of the coordinate data corresponding to thedifferent steering angle of the plurality of steering angles in avicinity of the target steering angle.
 5. The image signal processingapparatus according to claim 4, wherein the target coordinate datacorresponding to the target steering angle is generated by linearinterpolation of the coordinate data corresponding to the differentsteering angle of the plurality of steering angles in the vicinity ofthe target steering angle, and wherein the memory is configured to storethe target coordinate data in one of the plurality of data setscorresponding to the target steering angle.
 6. The image signalprocessing apparatus according to claim 1, wherein, to generate theguideline formed by the combination of the number of first directionsegments and the number of second direction segments, the imageprocessing circuitry is further configured to generate one or morecurved lines to represent at least one of the number of first directionsegments or at least one of the number of second direction segments byquadratic curve approximation using the coordinate data from the dataset selected from the plurality of data sets, wherein the coordinatedata includes coordinates of at least two end points representing the atleast one of the number of first direction segments or the at least oneof the number of second direction segments, and generate one or morestraight lines to represent at least a second one of the number of firstdirection segments or at least a second one of the number of seconddirection segments by collinear approximation using the coordinate datafrom the data set selected from the plurality of data sets.
 7. The imagesignal processing apparatus according to claim 6, wherein one middlepoint is a point positioned in the middle between the at least two endpoints in a horizontal direction or a vertical direction.
 8. The imagesignal processing apparatus according to claim 1, wherein the image ofthe vehicle periphery is an image of a back region of a vehicle.
 9. Theimage signal processing apparatus according to claim 1, wherein theimage of the vehicle periphery is an image of a side region of avehicle.
 10. The image signal processing apparatus according to claim 1,further comprising: an image sensor configured to capture the image ofthe vehicle periphery.
 11. A system comprising: an imaging deviceconfigured to capture an image of a vehicle periphery; a steering angleinformation receiving circuitry configured to receive steering angleinformation via an in-vehicle network; and an image processing circuitryconfigured to retrieve a data set from a plurality of data sets based onthe steering angle information that is received, each data set of theplurality of data sets including coordinate data corresponding to adifferent steering angle of a plurality of steering angles, generate aguideline formed by a combination of a number of first directionsegments and a number of second direction segments based on the data setretrieved from the plurality of data sets, superimpose the guideline onthe image of the vehicle periphery, output the image with the guidelinethat has been superimposed to a display, and update the guideline basedon a change of the steering angle information, wherein a first end pointcoordinate of a first segment of the number of first direction segmentsis the same coordinate as a first end point coordinate of a firstsegment of the number of second direction segments, wherein a second endpoint coordinate of the first segment of the number of first directionsegments is the same coordinate as a first end point coordinate of asecond segment of the number of first direction segments, and wherein asecond end point coordinate of the first segment of the number of seconddirection segments is the same coordinate as a first end pointcoordinate of a second segment of the number of second directionsegments.
 12. The system according to claim 11, wherein, to acquire thesteering angle information via the in-vehicle network, the steeringangle information receiving circuitry is further configured to acquire,repeatedly at a time interval, the steering angle information of asteering wheel.
 13. The system according to claim 11, furthercomprising: a memory configured to store the plurality of data sets,each data set of the plurality of data sets includes coordinate datacorresponding to the different steering angle of the plurality ofsteering angles.
 14. The system according to claim 13, wherein thememory is further configured to store the coordinate data of the eachdata set of the plurality of data sets corresponding to a discrete valueassociated with the different steering angle of the plurality ofsteering angles, and wherein the steering angle information receivingcircuitry is further configured to acquire the steering angleinformation of a target steering angle, and generate target coordinatedata corresponding to the target steering angle by interpolationprocessing of the coordinate data corresponding to the differentsteering angle of the plurality of steering angles in a vicinity of thetarget steering angle.
 15. The system according to claim 14, wherein thetarget coordinate data corresponding to the target steering angle isgenerated by linear interpolation of the coordinate data correspondingto the different steering angle of the plurality of steering angles inthe vicinity of the target steering angle, and wherein the memory isconfigured to store the target coordinate data in one of the pluralityof data sets corresponding to the target steering angle.
 16. The systemaccording to claim 11, wherein, to generate the guideline formed by thecombination of the number of first direction segments and the number ofsecond direction segments, the image processing circuitry is furtherconfigured to generate one or more curved lines to represent at leastone of the number of first direction segments or at least one of thenumber of second direction segments by quadratic curve approximationusing the coordinate data from the data set selected from the pluralityof data sets, wherein the coordinate data includes coordinates of atleast two end points representing the at least one of the number offirst direction segments or the at least one of the number of seconddirection segments, and generate one or more straight lines to representat least a second one of the number of first direction segments or atleast a second one of the number of second direction segments bycollinear approximation using the coordinate data from the data setselected from the plurality of data sets.
 17. The system according toclaim 11, wherein the image of the vehicle periphery is an image of aback region of a vehicle.
 18. The system according to claim 11, whereinthe image of the vehicle periphery is an image of a side region of avehicle.
 19. The system according to claim 15, wherein one middle pointis a point positioned in the middle between the at least two end pointsin a horizontal direction or a vertical direction.
 20. An imageprocessing method, the method comprising: acquiring an image of avehicle periphery; acquiring, with a steering angle informationacquisition circuitry, steering angle information from an in-vehiclenetwork; retrieving, with an image processing circuitry, a data set froma plurality of data sets based on the steering angle information that isacquired, each data set of the plurality of data sets includingcoordinate data corresponding to a different steering angle of aplurality of steering angles; generating, with the image processingcircuitry, a guideline formed by a combination of a number of firstdirection segments and a number of second direction segments based onthe data set retrieved from the plurality of data sets; superimposing,with the image processing circuitry, the guideline on the image of thevehicle periphery; outputting, with the image processing circuitry, theimage with the guideline that has been superimposed to a display; andupdating, with the image processing circuitry, the guideline based on achange of the steering angle information, wherein a first end pointcoordinate of a first segment of the number of first direction segmentsis the same coordinate as a first end point coordinate of a firstsegment of the number of second direction segments, wherein a second endpoint coordinate of the first segment of the number of first directionsegments is the same coordinate as a first end point coordinate of asecond segment of the number of first direction segments, and wherein asecond end point coordinate of the first segment of the number of seconddirection segments is the same coordinate as a first end pointcoordinate of a second segment of the number of second directionsegments.
 21. The image processing method according to claim 20, whereinacquiring the steering angle information from the in-vehicle networkfurther includes acquiring, repeatedly at a time interval, the steeringangle information of a steering wheel.
 22. The image processing methodaccording to claim 20, further comprising: acquiring, with the steeringangle information acquisition circuitry, the steering angle informationof a target steering angle; and generating, with the steering angleinformation acquisition circuitry, target coordinate data correspondingto the target steering angle by interpolation processing of thecoordinate data corresponding to the different steering angle of theplurality of steering angles in a vicinity of the target steering angle.23. The image processing method according to claim 22, whereingenerating the target coordinate data corresponding to the targetsteering angle further includes linearly interpolating the coordinatedata corresponding to the different steering angle of the plurality ofsteering angles in the vicinity of the target steering angle, andwherein the target coordinate data in one of the plurality of data setsis corresponding to the target steering angle.
 24. The image processingmethod according to claim 20, wherein generating the guideline formed bythe combination of the number of first direction segments and the numberof second direction segments further includes generating one or morecurved lines to represent at least one of the number of first directionsegments or at least one of the number of second direction segments byquadratic curve approximation using the coordinate data from the dataset selected from the plurality of data sets, the coordinate dataincluding coordinates of at least two end points representing the atleast one of the number of first direction segments or the at least oneof the number of second direction segments; and generating one or morestraight lines to represent at least a second one of the number of firstdirection segments or at least a second one of the number of seconddirection segments by collinear approximation using the coordinate datafrom the data set selected from the plurality of data sets.
 25. Theimage processing method according to claim 20, wherein acquiring theimage of the vehicle periphery further includes capturing, with an imagesensor, the image of the vehicle periphery.